The need to land aircraft, such as airplanes, helicopters, and spacecraft in zero/zero conditions is driving sensor fusion and computer vision systems for next-generation head-up displays. Safely landing the aircraft requires accurate information about the location of a target (e.g., runway). During an approach to a runway, the pilot must carefully control the navigation of the aircraft relative to a touchdown point. That is, pilots need to have a good situational awareness (e.g., heads up) of the outside world through heavy fog, smoke, snow, dust, or sand, to detect runways and obstacles on runways and/or in the approach path for a safe landing.
Advanced synthetic vision is a major focus of aerospace industry efforts to improve aviation safety. Some current research is focused on developing new and improved Enhanced Vision Systems. In these research efforts, there were various attempts to fuse sensor data from different modalities (based upon certified sensor availability) with synthetic vision platforms to provide pilots with additional features so that they can easily navigate to an airport, identify potential hazards, take avoidance action, and/or obtain sufficient visual reference of the runway.
The navigation data from a synthetic vision system (SVS) database is generated by many sources including, but not limited to, a differential global positioning system (DGPS), an inertial reference system (IRS), an attitude-heading reference system (AHRS), satellite and ground based devices (e.g., Instrument Landing Systems (ILS) and Microwave Landing System (MLS)). SVS modeling is advancing toward improving situational awareness in supporting pilots' ability to navigate in all conditions by providing information such as pitch, roll, yaw, lateral and vertical deviation, barometric altitudes and global positioning with runway heading, position, and dimensions. However, under low visibility conditions the pilot may not be able to visually verify the correctness of navigation data and the SVS database. Because SVS data is based on archived information (taken at earlier time than the time of the flight), the data can be impeded by updates into the scene, and thus, some cues may be missing from the actual data. In addition, navigation data cannot be used to navigate the aircraft to avoid obstacles on or near a runway because SVS models do not provide real-time information related to obstacles. Moreover, only a limited number of runways are equipped with adequate devices for providing accurate navigation attributes with the required accuracy to safely make low approaches and high-end equipment (e.g., ILS) is costly and is not available at all airports or to all runways at a particular airport.
Accordingly, there is a need to analyze real-time sensor imageries and fuse sensor data with SVS data to provide pilots with additional features so that they can easily navigate to the airport, identify potential hazards, take avoidance action, and obtain sufficient visual reference of a runway in real-time. As such, various embodiments of the present invention are configured to provide visual cues from one or more sensor images to enable a safe landing approach and minimize the number of missed approaches during low visibility conditions.